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Abstract : The purpose of this paper is to discuss development 
of a measure designed to assess student acquisition of 
communication competencies • This study assessed 180 students 
who took pre- and mid- program assessment tests, and the 
analysis of 33 questions taken from those tests* Specifically, 
the components examined included: critical thinking, 
interpersonal, decision-making, and theoretical competencies. 
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Multiple Approaches to Communication Program Assessment 
When our department began the state -mandated assessment 
process six years ago, we considered various assessment 
measures available (e.g. in Larson, Backlund, Redmond, and 
Barbour, 1978; and Rubin, Sisco, Moore, and Quianthy, 1983), 
but opted to develop our own. The dearth of funds for 
assessment, our concern about ti:e measures available, and the 
need for assessment relevant to our specific program were key 
factors in our decision to develop our own assessment testing. 

We gradually changed our emphasis from testing course 
content at graduation (Neer, 1989) to testing competencies 
throughout a student's studies, which changed the nature of the 
our entire assessment process (Aitken & Neer, 1992) . cur test 
has evolved into one with multiple approaches to data 
collection- -field specific content testing, self -report 
measures of communication measures, a portfolio, analysis of 
instruction, interviews, and more- -which are collected and 
analyzed at different stages of student development (Aitken & 
Neer, 1991) . Each student was involved in a minimum of three 
hours of assessment. To improve our original test for 
graduating seniors, we attempted to find ways to make our 
testing more effective by modifying tne questions to emphasize 
behavioral knowledge rather than concepts or labels, and by 
examining a student's communication consumption and activ. ties 
in various contexts. This multiple approach has proven useful 
in our attempt to collect accurate information. 
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Wg concur with those who suggest that assessment must be 
behaviorally- based, such as that conducted by Rubin's (1982) 
CCAI and the National Teacher Examination administered by the 
Educational Testing Service , However, we also concur with 
those who remind us that assessment should not emphasize 
technique over content {Hunt, 1990) and that cognitive learning 
also should be used to assess a learner's understanding of 
technique (Litterst, 1990). Thus, we administered a cognitive 
test to diagnose conceptual and theoretical deficiencies which 
may influence the student's behavioral performance or 
philosophical appreciation of communication. 

The purpose of this paper is to repor^ the results of 
assessing specific competencies. Our competency -based 
curriculum must have ways to measure student development of 
their communication competencies. Thus, we created an 
assessment measure that gave faculty information about the 
cognitive foundations of communication competencies. Although 
we obtained some valuable information from our original test 
results, with only a post-test, our information was limited. 
Thus, we wanted to develop a pre- test that could be 
administered to students upon entering the program, and again 
during the program. The first pre-and mid-program tests were 
given in August, 1991 and again in Jani:ary 1992. In this 
study, 180 students participated in pre and mid-program 
assessm.ent cognitive tests that contained 33 questions designed 
to measure critical thinking, interpersonal, dec is ion -making, 
and theoretical competencies. 
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Operationalization of Communication Coit^petency Test Components 

The faculty limited the Pre -Communication Assessment 
Measure (Pre-CAM) to the four competencies- -critical thinking, 
interpersonal, decision-making, and theoretical competencies-- 
common to two of five required courses (i.e., students must 
complete public speaking and introduction to communication 
theory prior to enrolling in upper -divisional coursework) . The 
need for developing such competencies has been supported by 
scholars in our field {^t.g. Frey & Botan, 1988; Fritz and 
Weaver, 1986; Spitzberg & Hurt, 1987; Wiemann, 1977). Many 
competencies are assessed at mid-program through portfolio 
evaluation of samples of a student's written and other assigned 
work (Aitken, 1993) . The department opted for a portfolio as a 
means of assessing actual communication performance because the 
Pre- and Post -CAM only were intended to measure cognitive 
performance. Although the competencies overlap, for our 
analysis, they were considered discrete categories or test 
components . 

Critical thinking competence (CT) , Critical thinking 
competence includes the ability to analyze supporting 
materials, make connections and applications to various 
contexts, and understand the logic of different thinking 
patterns. "Vhe student should recognize the following process 
skill objectives: (a) to increase accurate observation and 
memory, (b) to reflect on one's biases and perspectives, (c) to 
develop the ability to see various sides of an issue, (d) to 
increase objectivity, (e) to recognize persuasive language, (f) 
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to analyze premises and conclusions, (g) to recognize 
fallacious reasoning, (h) to determine important questions to 
ponder, and (i) to find answers to important questions through 
independent research and problem- solving skills. 

Interpersonal competence (rc) . a summary of the research 
on interpersonal communication includes ten major areas needing 
competence: self -disclosure, empathy, social relaxation, 
assertiveness, interaction management, altercentrism, 
expressiveness , supportiveness , immediacy, environmental 
control (Rubin & Nevins, 1988), Within this framework, the 
student should understand the influence of intrapersonal 
variables, including: perceptual processes, self -esteem and 
self-confidence, and belief -attitude-value structures. The 
student also should recognize the influential role that 
intrapersonal processes play in affecting interpersonal 
outcomes , 

DeciBion-makina competenp e (DM) . Decision-making 
coir^etence includes application of: reflective thinking 
processes, rhetorical sensitivity, argumentation methods, 
decision emergence, task process activities, relational 
activities, topic focus, listening, critical thinking, and 
developmental processes. The student should be able to 
determine the most appropriate methods by v/hich to communicate 
effectively, while applying various communication competencies 
to the decision-making process. 

Theoretical competence (t^) , Tlieoretical competence 
contains the ability to acknowledge the functions of theories 
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that organize, explain, and describe experience. The student 
should be able to appreciate going beyond the observable to 
provide predictive knowledge (to control events) and to 
stimulate and guide research in the field of communication. 
The student should be able to identify major paradigms of 
communication theories within differing contexts. 
Method 

Subj ects. The subjects were 180 students enrolled in a 
mid-sized, urban, mid-western university. Pre-program 
assessment was investigated by administering the Pre- CAM to 88 
students enrolled in the department's cornerstone course in 
communication theory. The course is required prior to student 
declaration of a communication major. The course 
("Introduction to the Study of Hxoman Communication") was 
developed to introduce students to the discipl-'ne and included 
prominent areas of incjuiry, theoretical frameworks, and methods 
of communication research. Mid-program assessment was 
conducted through administration of the Pre -CAM to 92 students 
enrolled in three upper-divisional courses in interpersonal, 
organizational, and group communication. Students completed 
the test during the 1991-1992 school year (first year students, 
sophomores, juniors, and seniors) . 

A pparatus . The Pre- CAM consisted of 33 multiple- choice 
items over the four competencies previously described. Fifty 
items were originally developed for the test; seventeen of the 
items were removed to improve the reliability of the test (see 
results section) . Approximately one -third of the test items 



Assessment 7 



examined conceptual generalizations underlying the human 
communication process. Another fifty percent of the items were 
situational- specific and required the application of 
communication concepts and principles in highly contextualized 
settings. The remaining items tested general knowledge of 
strategies and methods across more generalized communication 
contexts. Students not only had to understand the theoretical 
principles at hand, but to make decisions regarding which 
theory was most appropriate in each of the situations defined. 

Communication consumption measures > Communication 
consijmption was operationalized as frequency of communication 
activity engaged in both print and electronic media. Regarding 
newspaper readership, students identified those sections of the 
paper most frequently read. Print media also included serials 
most often read by type (e.g., news weeklies, arts and 
entertainment, general interest) . 

Communication actlvii-Y measures. Students were instructed 
to select their preferred communication activity along several 
dimensions: (a) the preferred interpersonal activity (dyadic 
vs. social interaction), and (b) the level of communication 
(e.g., interpersonal, small group, public, and mass) that they 
perceived to be most important. 

Additional mediators. The initial development of the CAM 
(Neer, 1989) revealed that two key factors that mediated test 
scores: (a) transfer status of students and (b) employment 
status while attending school.. Two additional variables were 
examined as potential mediators of cognitive performance on the 
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Pre-CAM. The first mediator was the kind of course assignment 
that students "most preferred" to complete (options included 
term paper, objective test, essay test, public speech, conduct 
a small group discussion, roleplay, perform in a skit) . These 
were reduced to written assignment, oral assignment, and 
interpersonal assignment to ensure adequate cell sizes across 
the various options. 

Hunt (1990) asserted the need for ensuring student 
motivation when he described the situation as his university. 
That is, to counter student resistance to taking assessment 
tests the university can provide a monetary incentive for a 
student to take the test and do well on the test. Our 
university has not offered a similar incentive; but we wanted 
to test whether an alternative reward incentive could increase 
motivation. Thus, the second variable tested for its mediating 
potential was incentive value or reward value for taking the 
Pre-CAM. Although the university has mandated assessment 
testing, our experience over the last five years has shown that 
seme students lack motivation to show up to take the test, 
others rush through the test in less than half the time most 
students require, and some students simply decide not to 
complete certain portions of the test. Although these students 
account for only fifteen percent of all students, their non- 
involvement has forced us to eliminate them from assessment 
analysis. We therefore decided to offer an incentive value to 
students who completed the Pre-CAM: half of the cornerstone 
courses (n = 2) were informed that the highest scores (i.e.. 
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one to three students) could exempt students from taking the 
final exam in the course. The other half of the cornerstone 
courses were not provided this offer, and only were informed 
that the test was required for the portfolio of all incoming 
majors. 

Analysis. The reliability of the Pre-CAM was assessed 
with KR-20 estimates and intracorrelations among the four 
components and correlation between each component and the test 
composite (i.e., the four summed components). We wanted to 
predict overall Pre- CAM composite scores and, more importantly, 
detect test components that contributed the most variance to 
composite scores or identify those components that yielded 
significance independent of the composite test score. Thus, 
ANOVA tested for significant mean differences between levels of 
all predictor variables. The Pre-CAM composite and MANOVA 
determined mean significance between the predictors and the 
four competencies (test components) that comprise the composite 
score. 
Results 

Pre-CAM reliability. The four competency- component Pre- 
CAM assessment test yielded an overall KR-20 reliability 
coefficient of .71. Irxdividual alphas for the components were 
as follows: theoretical competence = .56, decision-making 
competence = .63, critical thinking competence = .7 0, and 
interpersonal competence = .69. All four components correlated 
between .68 and .82 with the Pre-CAM composite (p < .001 with 
two-tailed test) . The test components were significantly 

ERLC 



Assessment 10 



intracorrelated with coefficients between .26 and .58 (p < .01 
with two- tailed test) with the theoretical competence component 
generating the highest intracorrelations (i»e., .40 to .58) 
with the other three components. Descriptive statistics 
revealed a grand mean of 20.59 and a standard deviation of 4.91 
with a scoring range of 23 (6 through 29) . 

Reliability of the Pre-CAM also was assessed in relation 
to self -reported grade point average (GPA) and class standing. 
If the Pre-CAM is a valid cognitive measure of communication 
competence, it could be argued that GPA and class standing 
should positively predict the Pre-CAM components; that is, 
because GPA is a general indicator of learning, GPA should 
function as an indicator of learning within and across 
coursework. Class standing also should function in a similar 
manner; that is, one would expect upper- divisional students-- 
who have completed courses in which the competencies have been 
developed- -to score higher on the Pre-CAM than incoming 
sophomore- level students . 

ANOVA significance was declared for the Pre-CAM composite 
with GPA (F = 6.75, df = 2.175, eta-squared = .07, power = .91, 
p < .001) . GPA also impacted on all competencies with the 
exception of critical thinking competence. GPA generated the 
strongest influence on decision-making competence (F = 5.40, 
eta- squared = .06, power = .84, p < .005) and theoretical 
comf itence (F = 4.74, eta-squared = .05, power = .79, p < .01). 
Although demonstrating a low power estimate, GPA also impacted 
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interpersonal competencG (F = 2.99, eta- squared = .03, power = 
.57 , p < .05) . 

Class standing failed to yield significance with the Pre- 
CAM composite but did yield MANOVA significance with the four 
comp3tencies (Wilks' = .91, df = 2.174, effect size = .05, 
power = .83, p < .02). Univariate significance was observed 
for the interpersonal competence component (F = 3.17, eta- 
squared = .04, power = .60, p < .04) with the theoretical 
competence and decision-making competence components each 
approaching significance at the .07 and .09 level, 
respectively. Table 1 reports mean scores for the influence of 
GPA and class standing on each of the four test components. 
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Effects of Communication Co nsumption on Communication 
Competenoe. MANOVA significance was observed with two 
consumption variables: frequency of weekly television viewing 
(Wilks' = .91, F = 2.12, df = 2.175, effect size = .05, power = 
.85, p < .03) and frequency of newspaper editorial page 
consumption (Wilks' = .94, F = 2.90, df = 1.173, effect size = 
.06, power = .78, p < .02). Univariate tests for frequency of 
television use revealed that students who viewed television 20 
or more hours per week scored lower on the theoretical 
competence component (F = 4.03, eta- squared = .04, power = .71, 
p < .02) and the interpersonal competence component (F = 5.64, 
eta-aquared = .06, power = .86, p < .004). Editorial page 
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reading resulted in univariate significance with the 
theoretical competence component (F = 8.27, eta- squared = .05, 
power = .86, p < .005) . Table 2 reports mean scores of 
television viewing and editorial reading frequency for the four 
test components. 
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Several other communication consumption variables 
approached MANOVA significance with the four test components. 
Before detexTnining whether to report these findings, AKTOVA 
first was conducted with the Pre- CAM composite. Our concern 
rested with identifying which Pre- CAM components influenced the 
Pre-CAM composite. Thus, ly^ANGVA tests that approached 
significance are reported for those consumption variables that 
yielded a significant ANOVA with the Pre-CAM composite. 

One consumption variable yielded ANOVA significance wich 
the Pre-CAM composite. The communication mediiom perceived to 
be the "most informative" yielded significance with the Pre-CAM 
composite (F = 4.61, df = 1.17 0, effect size = .026, power = 
.57, p < .03) and approached MANOVA significance with the four 
test ccmpo. lentG (Wilks'= .94, F = 2.26, eta- squared = .05, 
power - c p < .06). Univariate significance was observed 
will the decision-making component (F = 4.46, eta-squared = 
.03, power = .55, p < .04; interpersonal = 4.24, public/mass = 
4,77) . 
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The communication mediiam that students perceived as "most 
enjoyed" approached significance with the test composite (F = 
3.53, df = 1.17 8, eta- squared = .02, pov.^er = .46, p < .06) but 
failed to achieve MANOVA significance with the four test 
components. However, ANOVA significance was observed with the 
interpersonal competence component (F = 6.54, eta- squared = 
.04, power = .72, p < .01; interpersonal = 5.91, public/mass = 
5.03) . Students who preferred dyadic communication to other 
forms of interpersonal communication (i.e., social gatherings 
and activities) also scored higher on the interpersonal 
competence component (F = 4.93, df = 1.123, eta -squared = .04, 
power = .59, p < .03; dyadic = 6.14, other = 5.52). In 
addition, students who watched weekly news shows scored higher 
on the critical thinking component than students who failed to 
view weekly news shows (F = 4.34, df = 1.17 3, eta- squared = 
.02, power = .54, p < .04; watch = 4.29, do not watch = 3.7 6). 

These latter findings for which only univariate 
significance was observed constitutes "data snooping" on our 
part. The reader should interpret these findings with caution 
because the results failed to achieve either ANOVA significance 
with the test composite or MANOVA significance with the four 
test components. We report these findings only in the interest 
of informing educators and administrators on the potential 
influence of communication consumption and communication 
activity preferences on cognitive performance. 

Additional Pre-CAM mediators. In addition to the 
reliability variables and the communication consumption 
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variables, employmGnt status,- transfer vs. non-transfer status, 
preferred course assignment, and test incentive or reward value 
for completing the test also were examined. 

Reward value yielded significance with the test components 
(Wilks- = .88, F = 2.59, df = 1.88, effect size = .11, power = 
.71, p < .05). Although achieving manOVA significance, two of 
the test components only approached univariate significance. 
Students provided with the test incentive- -possible exemption 
from the course final examination- -scored higher on the 
theoretical competence (F = 2.83, eta- squared = .03, power = 
.48, p < .09; Reward = 6.31, No Reward = 5.75) and the 
interpersonal competence components (F = 3.17, eta- squared = 
.04, power = .52, p < .08; Reward = 5.84, No Reward = 5.18). 

Transfer status- -whether students had transferred into the 
university from another institution- -also yielded MANOVA 
significance (Wilks- = .89, F 2.45, df = 2.174, effect size = 
.06, power = .91, p < .02) and univariate significance with all 
test components except interpersonal competence. Significance 
also was observed with the test composite (F = 8.97, effect 
size = .09, power = .97, p = .001; community college transfer 
= 18.75, four-year college transfer = 21.78, continuing student 
= 21.60). Community college transfer students also scored 
lower on theoretical competence (F = 4.19, eta -squared = .05, 
power = .73, p < .02), decision-making competence (F = 5.37, 
eta- squared = .06, power - .84, p < .005) and critical thinking 
competence (F = 6.02, eta-squared = .02, power = .88, p < .003) 
than did either four-year college and university transfer 
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Students or students who only have attended the institution at 
which this study was conducted. Community college transfer 
students scored nearly a point lower than all other students on 
critical thinking competence (i.e., 3.36 versus 4.34) and over 
a half -point lower on both the theoretical competence (i.e., 
5.7 8 versus 6.60) and decision-making competencies (i.e., 4.17 
versus 4 .90) . 

Preferred course assignment (on which students thought 
they would perform the best) also approached MANOVA 
significance (Wilks' = .91, F = 1.90, df = 2.165, effect size = 
.045, power = .80, p < .058) and univariate significance with 
the theoretical competence component (F == 3.53, eta- squared == 
.04, power = .65, p < .03; written assignment = 6.24, oral 
assignment = 6.04, interpersonal assignment = 7.03). 
Discussion 

A basic and crucial outcome of this study is that 
apparently this form of assessment testing can work. Testing 
by individual components yields information about what entering 
students may need in their program and what exiting students 
have achieved. Despite the need to continue to modify the 
testing procedures to better measure the competencies, the 
method appears viable. Although assessment testing may raise 
more questions than it answers, the collection of data in this 
case has yielded sufficient information to allow the faculty to 
gain insights into the effectiveness of the program. We can 
support our contention that students are retaining information 
they learn in earlier courses. The significant difference for 
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Class standing on the interpersonal component , for example, 
provides useful ammunition for a long-time argument. Students 
sometimes claim that courses in interpersonal communication are 
"just common sense, " and they fail to change student 
effectiveness in interpersonal situations. Perhaps the 
significantly higher Pre- CAM scores on the interpersonal 
component for seniors helps negate the common sense assumption 
from students. This information may be valuable in defusing 
this myth when students enter the program and before they they 
have enrolled in the upper- divisional courses in interpersonal 
and group communication. 

Linking testing directly to communication consumption has 
excellent potential for analysis. The analysis of a student's 
communication consuirption and preferred activities outside the 
classroom provides a more interactive method for showing 
competency level. Although the communication consumption of 
our students failed to consistently affect Pre-CAM scores; 
knowledge about both consiamption- -the way students use media 
and the communication activities in which they engage- -can 
inform classroom teachers about how to relate course material 
to students' interests, tap their reservoir of knowledge during 
class discussion, and stimulate students to raise their own 
questions in class. This kind of information may provide ways 
to adapt course content to students and motivate students to 
redirect part of their communication consumption and activities 
to sources that strengthen their communication competencies. 
Although analysis of communication consumption and activity has 
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predictive value, one cannot assuine that a relationship 
indicates causation. Faculty can explore, however, the effect 
of using instructional materials to which the students easily 
relate. In addition, faculty can encourage use of media 
consumption and communication activities that may lead to 
higher competency levels. We are not prepared to outline all 
that should be done on this issue, but this kind of assessment 
can lead to improved student communication competencies. For 
instance, the viewing of weekly television news shows has a 
small but positive effect on critical thinking competence. 
Thus, in a course in rhetorical criticism, for example, 
students could be assigned to analyze a weekly news show or an 
editorial. Our intention is not to alter communication 
consumption habits- -although future research may suggest 
whether that is appropriate or inappropriate- -but to redirect 
some of the time that students spend on television viewing to 
watch programs that will reinforce and extend classroom 
learning . 

A small, but consistent, problem in our assessment testing 
has been student motivation. We have made participation 
voluntary because we want to avoid sabotaged results. While 
some departments have chosen to require certain test scores to 
graduate, we believe there are ethical problems in doing so 
when students had no such requirements upon entering the 
program. Although such a mandate may be appropriate at this 
point, doing so may change the "helping- faculty" nature of 
assessment we now enjoy to a more "punishing- students" nature. 
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An award for the highest student score or other positive 
approaches may prove more beneficial in obtaining the 
information needed to assess our program adequately (e.g., a 
citation on student's diploma and a certificate from the Chair 
and Dean that acknowledges this level of achievement) . 
Students may find this recognition useful to cite on their 
resumes. The use of a grade incentive- -possible final 
examination exemption in this case- -has shown that external 
motivational devicOsS help increase student performance on 
assessment tests. This information is valuable because faculty 
and administrators need assessment testing to yield the most 
accurate results possible. 

Perhaps it is not the scores, but the entire process of 
assessment- -which commences with the Pre-CAM- -that is most 
valuable because the process forces continual ^ elf -examination 
by the faculty. Twenty years ago, the trend of requiring 
student evaluation of faculty began a process of self -analysis 
of teaching that is well -accepted today. Perhaps the 
assessment trend is extending the depth of faculty self- 
analysis, while providing real potential for improved 
instruction. The assessment process and resulting changes can 
create a new sense of pride among administrators, faculty, and 
students. Our specific program is more clearly focused, 
faculty are engaged in continuous dialog about the improvement 
of undergraduate instruction, and students are responding to 
our improved dialog about their learning. While students may 
sometimes complain about the time-consuming nature of the 
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assessment process, they appear to take the tests seriously, 
show interest in the results, and appreciate our efforts to 
improve their learning. The assessment process has enabled 
faculty to see a larger picture of how their instruction fits 
into a student's entire learning process, v/hile enabling 
students to better understand faculty expectations. In our 
situation, there is a renewed emphasis on our joint 
responsibility for success. 
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Assessment 22 



Tablfi 1 

Effects of OPA a nd Standing 



ERIC 



Component Mean Scores 





TC 


DM 


CT 


IC 


GPA: 










2.00-2.25 


5.70 


3 .70 


3 . 10 


4 . 80 


2.26-2.99 


6.00 


4.48 


3.98 


5.70 


3 .00-3.75 


6.69 


4 .94 


4.15 


6.06 


Class Sr.andina: 










Sophomore 


5.81 


4 .42 


4.12 


5.31 


Junior 


6.27 


4.90 


3.98 


5.70 


Senior 


6.62 


4 .44 


4.00 


6.20 



Assessment 23 



Table 2 

Effects of Medi a Consnmption 



TC 



Component Mean Scores 
DM CT IC 



Editorial Pag e: 
Read 

Do not read 



6.75 
6.05 



4.67 
4.67 



4.23 
3.92 



6.02 
5.68 



Weekly Television Use 

Less than 3 hours 6.11 4.7 2 4.00 6.50 

Between 3 and 15 hours 6.68 4.74 4.08 5.89 

Twenty or more hours 5.47 4.13 3.65 4.87 
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